Species distribution modeling in R
C.H. Ross
2024-08-28
Brief intro to R
# Load libraries
library(tidyverse)
library(robis) # retrieves open source species occurrence data
library(biomod2) # species distribution modeling package
library(terra)
library(raster)# R basics ----
# Assignment operators
x <- 2 # preferred
x = 2 # also works
# Math operations
x * 2## [1] 4x^2 == x**2## [1] TRUEx/10## [1] 0.21e+12## [1] 1e+12log(pi)## [1] 1.14473log10(pi)## [1] 0.4971499# Useful tip -- pipes
# create some data
x <- c(1, 2, 3, 4, 5, 6) # Create a vector
x <- 1:6 # alternate notation
# Compute mean of x
mean(x)## [1] 3.5# Alternative with piping
x |> mean()## [1] 3.5# Data frame example with made up data
fake.zoop.data <- data.frame("Genus" = c("Calanus", "Temora", "Centropages"),
"Species" = c("finmarchicus", "longicornis", "typicus"),
"Abundance" = c(250, 450, 500),
"NumAdults" = c(175, 300, 450),
"MeanDryWeight" = c(17, 10, 8),
"StErrDryWeight" = c(3.4, 1.6, 0.5))
# View data frame
View(fake.zoop.data)
# Accessing data frame columns
fake.zoop.data$Genus## [1] "Calanus" "Temora" "Centropages"# Accessing multiple columns
fake.zoop.data[c("Genus", "Species")]## Genus Species
## 1 Calanus finmarchicus
## 2 Temora longicornis
## 3 Centropages typicus# Add a column (base R)
fake.zoop.data$NumJuvenile <- c(75, 150, 50)
# Modifying a column
fake.zoop.data$NumJuvenile <- fake.zoop.data$Abundance - fake.zoop.data$NumAdultsWe can do similar data manipulation using more intuitive methods from the dplyr package. The dplyr package is part of the larger tidyverse, which is also worth familiarizing yourself with if you program in R a lot. There are many helpful cheat sheets available. Here are a few examples below:
# Add a column to original data frame
fake.zoop.data <- fake.zoop.data |>
dplyr::mutate(SampMethod = c("Vertical", "Vertical", "Vertical"))
fake.zoop.data## Genus Species Abundance NumAdults MeanDryWeight StErrDryWeight
## 1 Calanus finmarchicus 250 175 17 3.4
## 2 Temora longicornis 450 300 10 1.6
## 3 Centropages typicus 500 450 8 0.5
## NumJuvenile SampMethod
## 1 75 Vertical
## 2 150 Vertical
## 3 50 Vertical# Filter the data frame
one.genus <- fake.zoop.data |>
dplyr::filter(Genus == "Calanus" )
one.genus## Genus Species Abundance NumAdults MeanDryWeight StErrDryWeight
## 1 Calanus finmarchicus 250 175 17 3.4
## NumJuvenile SampMethod
## 1 75 Vertical# Can combine these functions calls in a pipe
one.species <- fake.zoop.data |>
dplyr::mutate(SampMethod = c("Vertical", "Vertical", "Vertical")) |>
dplyr::filter(Genus == "Calanus" ) |>
dplyr::select(Genus, Abundance, SampMethod)
one.species## Genus Abundance SampMethod
## 1 Calanus 250 VerticalNext, we’ll plot the fake data in base R.
# Plot the data
plot(x = fake.zoop.data$Abundance, y = fake.zoop.data$MeanDryWeight)
The base R plot function works fine, but there are also plotting packages with a lot more flexibility and features.
One example is the ggplot2 package, also from the tidyverse:
# A better way to plot the data
ggplot(data = fake.zoop.data, mapping = aes(x = Abundance, y = MeanDryWeight)) +
geom_point()
# Alternative notation
ggplot() +
geom_point(data = fake.zoop.data, mapping = aes(x = Abundance, y = MeanDryWeight))
# Add error bars
ggplot(data = fake.zoop.data, mapping = aes(x = Abundance, y = MeanDryWeight)) +
geom_point() +
geom_errorbar(mapping = aes(ymin = MeanDryWeight-StErrDryWeight, ymax = MeanDryWeight+StErrDryWeight))
# Improve plot aesthetics
ggplot(data = fake.zoop.data, mapping = aes(x = Abundance, y = MeanDryWeight)) +
geom_point() +
geom_errorbar(mapping = aes(ymin = MeanDryWeight-StErrDryWeight, ymax = MeanDryWeight+StErrDryWeight)) +
labs(x = bquote("Abundance (individuals"~m^{-3}*")"), y = "Mean Dry Weight (mg)") +
theme_bw() # many theme options
Species distribution modeling in R
Species distribution models (SDMs) are an important tool used often in ecology to model habitat suitability for a given species. There are many ways to produce SDMs in R. Below is a simple example using the package biomod2, which has the capability to simultaneously produce multiple types of models and build ensembles. This package can be a good introduction to SDMs if you are new to coding and/or modeling. Biomod2 can produce models using up to 11 different algorithms. It runs the models behind the scenes using the respective R packages, which makes it a great package for introducing SDMs and quickly comparing different models using relatively little code.
The biomod2 github page is a really helpful resource: [Ensemble Platform for Species Distribution Modeling][https://biomodhub.github.io/biomod2/]
Fetch species data
The first step is fetching the species data. For this example, I will be using right whale occurence data from the Ocean Biodiversity Information System (OBIS), which is a global open access resource for marine biodiversity data and information.
# Fetching species data
RightWhale <- robis::occurrence("Eubalaena glacialis", # Right whale
startdate = as.Date("2005-01-01"),
enddate = as.Date("2015-12-31"))## Retrieved 1549 records of approximately 1549 (100%)Pulling records from OBIS for right whales between 2005 and 2015 resulted in 1,549 data points. Next, we will visualize our species data using ggplot2.
# Load world map data
worldmap <- ggplot2::map_data("world")
# Plot occurrences on world map
ggplot2::ggplot(data = RightWhale, mapping = aes(x = decimalLongitude, y = decimalLatitude)) +
# Add occurrence data
geom_point() +
# Add map data
geom_polygon(data = worldmap, aes(long, lat, group = group), fill = NA, colour = "gray43") +
# Sets map bounds to geographic range of the species data
coord_quickmap(xlim = c(round(min(RightWhale$decimalLongitude)),
round(max(RightWhale$decimalLongitude))),
ylim = c(round(min(RightWhale$decimalLatitude)),
round(max(RightWhale$decimalLatitude))),
expand = TRUE) +
# Clean up theme
theme_bw() +
theme(panel.grid.major = element_blank(), panel.grid.minor = element_blank())
We can see that the data are largely concentrated along the east coast of the United States, with one point in the Azores. Since the environmental data we will use for modeling are regional and only for the summer, we need to crop the data to that bounding box and select only data from July through September.
# Define bounding box for data -- zoom in on Gulf of Maine
BB <- c(-75, -60, 35, 46)
# Crop data to bounding box
RightWhale <- RightWhale |>
# Shorten names
dplyr::rename(lon = decimalLongitude,
lat = decimalLatitude) |>
dplyr::mutate(year = as.numeric(year),
month = as.numeric(month)) |>
# Filter July, Aug, Sep
dplyr::filter(month %in% 7:9) |>
# Filter to bounding box
dplyr::filter(lon >= BB[1] & lon <= BB[2] &
lat >= BB[3] & lat <= BB[4]) |>
# Select relevant columns
dplyr::select(lon, lat, year)
# Plot again
ggplot2::ggplot(data = RightWhale, mapping = aes(x = lon, y = lat)) +
# Add occurrence data
geom_point() +
# Add map data
geom_polygon(data = worldmap, aes(long, lat, group = group), fill = NA, colour = "gray43") +
coord_quickmap(xlim = c(BB[1], BB[2]),
ylim = c(BB[3], BB[4]),
expand = TRUE) +
# Clean up theme
theme_bw() +
theme(panel.grid.major = element_blank(), panel.grid.minor = element_blank())
Loading environmetnal data
Next, we need to load some environmental data to use as predictors in our model.
# Load environmental predictors
# Monthly depth (static), SST (contemporaneous), and CHL (contemporaneous) for summer (jul-sep)
env_dat <- raster::brick("env_stack.tif")
# Set extent
extent(env_dat) <- extent(-82.92685,
-55.67632,
22.56053,
48.26303)
# Add names to raster stack -- were not saved in original tif -- workaround
vars <- c("Depth", "SST", "CHL")
years <- 2005:2015
names(env_dat) <- paste(rep(vars, times = 11), rep(years, each = 3), sep = ".")
env_dat## class : RasterBrick
## dimensions : 575, 491, 282325, 33 (nrow, ncol, ncell, nlayers)
## resolution : 0.05550006, 0.0447 (x, y)
## extent : -82.92685, -55.67632, 22.56053, 48.26303 (xmin, xmax, ymin, ymax)
## crs : +proj=longlat +datum=WGS84 +no_defs
## source : env_stack.tif
## names : Depth.2005, SST.2005, CHL.2005, Depth.2006, SST.2006, CHL.2006, Depth.2007, SST.2007, CHL.2007, Depth.2008, SST.2008, CHL.2008, Depth.2009, SST.2009, CHL.2009, ...
## min values : 1.16924000, 9.86632633, 0.04109016, 1.16924000, 13.36531162, 0.04563540, 1.16924000, 11.37914085, 0.04466756, 1.16924000, 12.12702751, 0.03868118, 1.16924000, 11.19710350, 0.04641888, ...
## max values : 5309.75977, 30.20734, 25.74244, 5309.75977, 29.67356, 27.01186, 5309.75977, 30.73302, 25.57668, 5309.75977, 29.92080, 23.06375, 5309.75977, 30.33197, 37.78852, ...# Crop to bounding box
env_dat <- crop(env_dat, extent(BB))
# Plot first depth layer
plot(env_dat$Depth.2005)
# Plot first SST layer
plot(env_dat$SST.2005)
# Plot first CHL layer
plot(env_dat$CHL.2005)
### Format model data
An important thing to consider when producing SDMs is what sort of response you want to model. For example, SDMs can model presence-only, presence/absence, and abundance data. Approaches can be extended further to model things like density, but that requires the survey design to meet certain assumptions.
The right whale data from OBIS does have an individual count column, but the data are from many different sources, and likely of varying quality. Due to these variations in the data, a presence-only or presence/absence approach would make the most sense. Presence-only limits the types of models that can be used (e.g., MaxEnt is a good option), whereas a presence/absence approach is more readily adaptable to different model types. Another option is to create “pseudo-absences,” which are essentially fake absence data sampled from background points on the environmental data grid. Biomod2 has the ability to produce pseudo-absences, allowing us to produce models using algorithms beyond MaxEnt.
The next step is to format our data for the biomod2 package with a pseudo-absence generation approach.
# Add presence/absence column to right whale data -- binary
RightWhale <- RightWhale |>
dplyr::mutate(pa = 1) # we will generate pseudo absences later
# Isolate binary presence/absence data - training years 2005-2010
trainingPA <- RightWhale |>
dplyr::filter(year < 2011) |>
dplyr::select(pa)
# Isolate presence/absence coordinates
trainingXY <- RightWhale |>
dplyr::filter(year < 2011) |>
dplyr::select(lon, lat)We also want to set some data aside for model evaluation. For this, we will use 2011-2014. Biomod2 will generate pseudo-absences for the training data, but I am not sure if it does that for the evaluation data. So here is a workaround sampling from the boundaries of the environmental data for background points.
# Select background points and filter out NAs
# Ideally absence data is available
background <- as.data.frame(env_dat$Depth.2005, xy = TRUE) |> # pull locations from environmental data grid
dplyr::rename(lon = x, lat = y) |>
sample_n(30) |>
dplyr::mutate(pa = 0, year = sample(c(2005, 2011), n(), replace = TRUE)) |>
dplyr::filter(!is.na(Depth.2005)) |>
dplyr::select(lon, lat, pa, year)
# Isolate binary presence/absence data - test years 2011-2014
evalPA <- RightWhale |>
rbind(background) |> # attach background points
dplyr::filter(year >= 2011 & year < 2015) |>
dplyr::select(pa)
# Isolate presence/absence coordinates
evalXY <- RightWhale |>
rbind(background) |> # attach background points
dplyr::filter(year >= 2011 & year < 2015) |>
dplyr::select(lon, lat)We need to subset the environmental data raster to select only layers before 2010. For simplicity, we will take a climatalogical approach to the model by using the average summer SST and CHL values across the training time period. This is definitely not best practice.
# Create climatology of environmental training data for summer 2005-2010
# There is definitely a better way to do this
env_train <- raster::subset(env_dat, c(grep('2005', names(env_dat)),
grep('2007', names(env_dat)),
grep('2008', names(env_dat)),
grep('2009', names(env_dat)),
grep('2010', names(env_dat))))
sst <- raster::subset(env_train, grep('SST', names(env_train))) |>
mean(na.rm = TRUE)
chl <- raster::subset(env_train, grep('CHL', names(env_train))) |>
mean(na.rm = TRUE)
depth <- env_train$Depth.2005 # static variable
env_train <- stack(depth, sst, chl)
names(env_train) <- varsDo the same for the evaluation environmental data
# Create climatology of environmental testing data for July 2011-2014
env_eval <- raster::subset(env_dat, c(grep('2011', names(env_dat)),
grep('2012', names(env_dat)),
grep('2013', names(env_dat)),
grep('2014', names(env_dat))))
sst.eval <- raster::subset(env_eval, grep('SST', names(env_eval))) |>
mean(na.rm = TRUE)
chl.eval <- raster::subset(env_eval, grep('CHL', names(env_eval))) |>
mean(na.rm = TRUE)
depth.eval <- env_eval$Depth.2011 # static variable
env_eval <- stack(depth.eval, sst.eval, chl.eval)
names(env_eval) <- varsNow that our data are ready to go, we will input them to the biomod2 data formatting function. We will try three different models: generalized linear models (GLMs), generalized additive models (GAMs), and random forests (RFs).
For a complete list of model options available in biomod2 and dependencies see model options vignette.
# Specify models to run
modelFormulas <- c('GLM', 'GAM', 'RF')
# Format data for use in Biomod2 modelling function & generate random pseudo-absences
biomodData <- BIOMOD_FormatingData(resp.var = trainingPA,
expl.var = env_train,
resp.xy = trainingXY,
eval.resp.var = evalPA,
eval.resp.xy = evalXY,
eval.expl.var = env_eval,
PA.nb.rep = 4,
PA.nb.absences = 200,
PA.strategy = "random",
resp.name = "RightWhale",
filter.raster = TRUE)##
## -=-=-=-=-=-=-=-=-=-=-=-=-= RightWhale Data Formating -=-=-=-=-=-=-=-=-=-=-=-=-=
##
## !!! Some data are located in the same raster cell.
## Only the first data in each cell will be kept as `filter.raster = TRUE`.
##
## Checking Pseudo-absence selection arguments...
##
## > random pseudo absences selection
## > Pseudo absences are selected in explanatory variables
##
## !!! Some data are located in the same raster cell.
## Only the first data in each cell will be kept as `filter.raster = TRUE`.
## ! No data has been set aside for modeling evaluation
## -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Done -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=biomodData##
## -=-=-=-=-=-=-=-=-=-=-=-=-=-= BIOMOD.formated.data -=-=-=-=-=-=-=-=-=-=-=-=-=-=
##
## dir.name = .
##
## sp.name = RightWhale
##
## 87 presences, 0 true absences and 795 undefined points in dataset
##
##
## 3 explanatory variables
##
## Depth SST CHL
## Min. : 2.583 Min. :11.65 Min. : 0.04995
## 1st Qu.: 88.737 1st Qu.:16.47 1st Qu.: 0.15971
## Median : 206.703 Median :20.22 Median : 0.40891
## Mean :1445.225 Mean :20.65 Mean : 0.92844
## 3rd Qu.:3005.230 3rd Qu.:25.17 3rd Qu.: 0.75132
## Max. :4783.216 Max. :28.49 Max. :25.38794
##
##
## Evaluation data :
##
## 26 presences, 4 true absences and 0 undefined points in dataset
##
##
##
## Depth SST CHL
## Min. : 43.43 Min. :12.69 Min. :0.1198
## 1st Qu.: 147.68 1st Qu.:16.62 1st Qu.:0.5835
## Median : 175.19 Median :16.88 Median :0.6138
## Mean : 397.74 Mean :17.91 Mean :0.8196
## 3rd Qu.: 221.80 3rd Qu.:19.01 3rd Qu.:0.6374
## Max. :4354.47 Max. :25.50 Max. :7.2483
##
##
## 4 Pseudo Absences dataset available ( PA1, PA2, PA3, PA4 ) with
## 200 (PA1, PA2, PA3, PA4) pseudo absences
##
## -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=Build models
We are now ready for the modeling stage.
# Build the models
modelOut <- BIOMOD_Modeling(bm.format = biomodData,
modeling.id = "RightWhale",
models = c("GLM", "GAM", "RF"),
CV.nb.rep = 5, # 5-fold cross validation
data.split.perc = 70, # 70%/30% training/testing data split
prevalence = 0.5,
var.import = 5,
metric.eval = c('ROC', 'TSS', 'KAPPA'),
scale.models = FALSE,
do.progress = TRUE)## Warning in .bm_ModelingOptions.check.args(data.type = data.type, models =
## models, : Only one GAM model can be activated. 'GAM.mgcv.gam' has been set
## (other available : 'GAM.gam.gam' or 'GAM.mgcv.bam')## Warning: executing %dopar% sequentially: no parallel backend registered## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurred
## Warning: glm.fit: fitted probabilities numerically 0 or 1 occurredModel evaluations
Evaluations for each model formula, evaluation, and estimate of pseudo-absences is available in the output model object.
# Extract model evaluations
modelEvals <- get_evaluations(obj = modelOut)
modelEvals## full.name PA run algo metric.eval cutoff
## 1 RightWhale_PA1_RUN1_GLM PA1 RUN1 GLM ROC 422.0
## 2 RightWhale_PA1_RUN1_GLM PA1 RUN1 GLM TSS 423.0
## 3 RightWhale_PA1_RUN1_GLM PA1 RUN1 GLM KAPPA 526.0
## 4 RightWhale_PA1_RUN1_GAM PA1 RUN1 GAM ROC 552.5
## 5 RightWhale_PA1_RUN1_GAM PA1 RUN1 GAM TSS 553.5
## 6 RightWhale_PA1_RUN1_GAM PA1 RUN1 GAM KAPPA 553.5
## 7 RightWhale_PA1_RUN1_RF PA1 RUN1 RF ROC 527.0
## 8 RightWhale_PA1_RUN1_RF PA1 RUN1 RF TSS 525.0
## 9 RightWhale_PA1_RUN1_RF PA1 RUN1 RF KAPPA 525.0
## 10 RightWhale_PA1_RUN2_GLM PA1 RUN2 GLM ROC 470.5
## 11 RightWhale_PA1_RUN2_GLM PA1 RUN2 GLM TSS 470.0
## 12 RightWhale_PA1_RUN2_GLM PA1 RUN2 GLM KAPPA 759.0
## 13 RightWhale_PA1_RUN2_GAM PA1 RUN2 GAM ROC 525.0
## 14 RightWhale_PA1_RUN2_GAM PA1 RUN2 GAM TSS 525.5
## 15 RightWhale_PA1_RUN2_GAM PA1 RUN2 GAM KAPPA 593.0
## 16 RightWhale_PA1_RUN2_RF PA1 RUN2 RF ROC 506.0
## 17 RightWhale_PA1_RUN2_RF PA1 RUN2 RF TSS 505.0
## 18 RightWhale_PA1_RUN2_RF PA1 RUN2 RF KAPPA 505.0
## 19 RightWhale_PA1_RUN3_GLM PA1 RUN3 GLM ROC 367.5
## 20 RightWhale_PA1_RUN3_GLM PA1 RUN3 GLM TSS 364.0
## 21 RightWhale_PA1_RUN3_GLM PA1 RUN3 GLM KAPPA 620.0
## 22 RightWhale_PA1_RUN3_GAM PA1 RUN3 GAM ROC 577.5
## 23 RightWhale_PA1_RUN3_GAM PA1 RUN3 GAM TSS 575.5
## 24 RightWhale_PA1_RUN3_GAM PA1 RUN3 GAM KAPPA 575.5
## 25 RightWhale_PA1_RUN3_RF PA1 RUN3 RF ROC 512.0
## 26 RightWhale_PA1_RUN3_RF PA1 RUN3 RF TSS 510.0
## 27 RightWhale_PA1_RUN3_RF PA1 RUN3 RF KAPPA 510.0
## 28 RightWhale_PA1_RUN4_GLM PA1 RUN4 GLM ROC 433.5
## 29 RightWhale_PA1_RUN4_GLM PA1 RUN4 GLM TSS 431.0
## 30 RightWhale_PA1_RUN4_GLM PA1 RUN4 GLM KAPPA 713.0
## 31 RightWhale_PA1_RUN4_GAM PA1 RUN4 GAM ROC 582.0
## 32 RightWhale_PA1_RUN4_GAM PA1 RUN4 GAM TSS 568.0
## 33 RightWhale_PA1_RUN4_GAM PA1 RUN4 GAM KAPPA 568.0
## 34 RightWhale_PA1_RUN4_RF PA1 RUN4 RF ROC 480.0
## 35 RightWhale_PA1_RUN4_RF PA1 RUN4 RF TSS 475.0
## 36 RightWhale_PA1_RUN4_RF PA1 RUN4 RF KAPPA 475.0
## 37 RightWhale_PA1_RUN5_GLM PA1 RUN5 GLM ROC 666.5
## 38 RightWhale_PA1_RUN5_GLM PA1 RUN5 GLM TSS 366.0
## 39 RightWhale_PA1_RUN5_GLM PA1 RUN5 GLM KAPPA 712.0
## 40 RightWhale_PA1_RUN5_GAM PA1 RUN5 GAM ROC 586.5
## 41 RightWhale_PA1_RUN5_GAM PA1 RUN5 GAM TSS 565.5
## 42 RightWhale_PA1_RUN5_GAM PA1 RUN5 GAM KAPPA 565.5
## 43 RightWhale_PA1_RUN5_RF PA1 RUN5 RF ROC 497.0
## 44 RightWhale_PA1_RUN5_RF PA1 RUN5 RF TSS 500.0
## 45 RightWhale_PA1_RUN5_RF PA1 RUN5 RF KAPPA 500.0
## 46 RightWhale_PA1_allRun_GLM PA1 allRun GLM ROC 364.0
## 47 RightWhale_PA1_allRun_GLM PA1 allRun GLM TSS 363.0
## 48 RightWhale_PA1_allRun_GLM PA1 allRun GLM KAPPA 716.0
## 49 RightWhale_PA1_allRun_GAM PA1 allRun GAM ROC 572.0
## 50 RightWhale_PA1_allRun_GAM PA1 allRun GAM TSS 574.0
## 51 RightWhale_PA1_allRun_GAM PA1 allRun GAM KAPPA 583.0
## 52 RightWhale_PA1_allRun_RF PA1 allRun RF ROC 508.0
## 53 RightWhale_PA1_allRun_RF PA1 allRun RF TSS 510.0
## 54 RightWhale_PA1_allRun_RF PA1 allRun RF KAPPA 510.0
## 55 RightWhale_PA2_RUN1_GLM PA2 RUN1 GLM ROC 496.5
## 56 RightWhale_PA2_RUN1_GLM PA2 RUN1 GLM TSS 496.0
## 57 RightWhale_PA2_RUN1_GLM PA2 RUN1 GLM KAPPA 496.0
## 58 RightWhale_PA2_RUN1_GAM PA2 RUN1 GAM ROC 543.5
## 59 RightWhale_PA2_RUN1_GAM PA2 RUN1 GAM TSS 537.0
## 60 RightWhale_PA2_RUN1_GAM PA2 RUN1 GAM KAPPA 537.0
## 61 RightWhale_PA2_RUN1_RF PA2 RUN1 RF ROC 491.0
## 62 RightWhale_PA2_RUN1_RF PA2 RUN1 RF TSS 493.0
## 63 RightWhale_PA2_RUN1_RF PA2 RUN1 RF KAPPA 493.0
## 64 RightWhale_PA2_RUN2_GLM PA2 RUN2 GLM ROC 356.5
## 65 RightWhale_PA2_RUN2_GLM PA2 RUN2 GLM TSS 355.0
## 66 RightWhale_PA2_RUN2_GLM PA2 RUN2 GLM KAPPA 458.0
## 67 RightWhale_PA2_RUN2_GAM PA2 RUN2 GAM ROC 558.5
## 68 RightWhale_PA2_RUN2_GAM PA2 RUN2 GAM TSS 556.0
## 69 RightWhale_PA2_RUN2_GAM PA2 RUN2 GAM KAPPA 556.0
## 70 RightWhale_PA2_RUN2_RF PA2 RUN2 RF ROC 532.0
## 71 RightWhale_PA2_RUN2_RF PA2 RUN2 RF TSS 530.0
## 72 RightWhale_PA2_RUN2_RF PA2 RUN2 RF KAPPA 530.0
## 73 RightWhale_PA2_RUN3_GLM PA2 RUN3 GLM ROC 479.0
## 74 RightWhale_PA2_RUN3_GLM PA2 RUN3 GLM TSS 547.0
## 75 RightWhale_PA2_RUN3_GLM PA2 RUN3 GLM KAPPA 686.0
## 76 RightWhale_PA2_RUN3_GAM PA2 RUN3 GAM ROC 543.5
## 77 RightWhale_PA2_RUN3_GAM PA2 RUN3 GAM TSS 521.0
## 78 RightWhale_PA2_RUN3_GAM PA2 RUN3 GAM KAPPA 544.0
## 79 RightWhale_PA2_RUN3_RF PA2 RUN3 RF ROC 504.0
## 80 RightWhale_PA2_RUN3_RF PA2 RUN3 RF TSS 505.0
## 81 RightWhale_PA2_RUN3_RF PA2 RUN3 RF KAPPA 505.0
## 82 RightWhale_PA2_RUN4_GLM PA2 RUN4 GLM ROC 392.5
## 83 RightWhale_PA2_RUN4_GLM PA2 RUN4 GLM TSS 395.0
## 84 RightWhale_PA2_RUN4_GLM PA2 RUN4 GLM KAPPA 395.0
## 85 RightWhale_PA2_RUN4_GAM PA2 RUN4 GAM ROC 508.5
## 86 RightWhale_PA2_RUN4_GAM PA2 RUN4 GAM TSS 507.0
## 87 RightWhale_PA2_RUN4_GAM PA2 RUN4 GAM KAPPA 507.0
## 88 RightWhale_PA2_RUN4_RF PA2 RUN4 RF ROC 538.0
## 89 RightWhale_PA2_RUN4_RF PA2 RUN4 RF TSS 540.0
## 90 RightWhale_PA2_RUN4_RF PA2 RUN4 RF KAPPA 540.0
## 91 RightWhale_PA2_RUN5_GLM PA2 RUN5 GLM ROC 434.5
## 92 RightWhale_PA2_RUN5_GLM PA2 RUN5 GLM TSS 435.0
## 93 RightWhale_PA2_RUN5_GLM PA2 RUN5 GLM KAPPA 554.0
## 94 RightWhale_PA2_RUN5_GAM PA2 RUN5 GAM ROC 504.0
## 95 RightWhale_PA2_RUN5_GAM PA2 RUN5 GAM TSS 505.5
## 96 RightWhale_PA2_RUN5_GAM PA2 RUN5 GAM KAPPA 505.5
## 97 RightWhale_PA2_RUN5_RF PA2 RUN5 RF ROC 507.0
## 98 RightWhale_PA2_RUN5_RF PA2 RUN5 RF TSS 509.0
## 99 RightWhale_PA2_RUN5_RF PA2 RUN5 RF KAPPA 509.0
## 100 RightWhale_PA2_allRun_GLM PA2 allRun GLM ROC 547.5
## 101 RightWhale_PA2_allRun_GLM PA2 allRun GLM TSS 540.0
## 102 RightWhale_PA2_allRun_GLM PA2 allRun GLM KAPPA 540.0
## 103 RightWhale_PA2_allRun_GAM PA2 allRun GAM ROC 539.0
## 104 RightWhale_PA2_allRun_GAM PA2 allRun GAM TSS 539.0
## 105 RightWhale_PA2_allRun_GAM PA2 allRun GAM KAPPA 539.0
## 106 RightWhale_PA2_allRun_RF PA2 allRun RF ROC 512.0
## 107 RightWhale_PA2_allRun_RF PA2 allRun RF TSS 514.0
## 108 RightWhale_PA2_allRun_RF PA2 allRun RF KAPPA 514.0
## 109 RightWhale_PA3_RUN1_GLM PA3 RUN1 GLM ROC 490.5
## 110 RightWhale_PA3_RUN1_GLM PA3 RUN1 GLM TSS 491.0
## 111 RightWhale_PA3_RUN1_GLM PA3 RUN1 GLM KAPPA 491.0
## 112 RightWhale_PA3_RUN1_GAM PA3 RUN1 GAM ROC 515.5
## 113 RightWhale_PA3_RUN1_GAM PA3 RUN1 GAM TSS 555.0
## 114 RightWhale_PA3_RUN1_GAM PA3 RUN1 GAM KAPPA 555.0
## 115 RightWhale_PA3_RUN1_RF PA3 RUN1 RF ROC 516.0
## 116 RightWhale_PA3_RUN1_RF PA3 RUN1 RF TSS 515.0
## 117 RightWhale_PA3_RUN1_RF PA3 RUN1 RF KAPPA 515.0
## 118 RightWhale_PA3_RUN2_GLM PA3 RUN2 GLM ROC 519.0
## 119 RightWhale_PA3_RUN2_GLM PA3 RUN2 GLM TSS 518.0
## 120 RightWhale_PA3_RUN2_GLM PA3 RUN2 GLM KAPPA 518.0
## 121 RightWhale_PA3_RUN2_GAM PA3 RUN2 GAM ROC 530.0
## 122 RightWhale_PA3_RUN2_GAM PA3 RUN2 GAM TSS 528.5
## 123 RightWhale_PA3_RUN2_GAM PA3 RUN2 GAM KAPPA 578.0
## 124 RightWhale_PA3_RUN2_RF PA3 RUN2 RF ROC 514.0
## 125 RightWhale_PA3_RUN2_RF PA3 RUN2 RF TSS 510.0
## 126 RightWhale_PA3_RUN2_RF PA3 RUN2 RF KAPPA 510.0
## 127 RightWhale_PA3_RUN3_GLM PA3 RUN3 GLM ROC 339.5
## 128 RightWhale_PA3_RUN3_GLM PA3 RUN3 GLM TSS 750.0
## 129 RightWhale_PA3_RUN3_GLM PA3 RUN3 GLM KAPPA 750.0
## 130 RightWhale_PA3_RUN3_GAM PA3 RUN3 GAM ROC 560.5
## 131 RightWhale_PA3_RUN3_GAM PA3 RUN3 GAM TSS 560.5
## 132 RightWhale_PA3_RUN3_GAM PA3 RUN3 GAM KAPPA 560.5
## 133 RightWhale_PA3_RUN3_RF PA3 RUN3 RF ROC 509.0
## 134 RightWhale_PA3_RUN3_RF PA3 RUN3 RF TSS 507.0
## 135 RightWhale_PA3_RUN3_RF PA3 RUN3 RF KAPPA 507.0
## 136 RightWhale_PA3_RUN4_GLM PA3 RUN4 GLM ROC 561.0
## 137 RightWhale_PA3_RUN4_GLM PA3 RUN4 GLM TSS 562.0
## 138 RightWhale_PA3_RUN4_GLM PA3 RUN4 GLM KAPPA 562.0
## 139 RightWhale_PA3_RUN4_GAM PA3 RUN4 GAM ROC 534.5
## 140 RightWhale_PA3_RUN4_GAM PA3 RUN4 GAM TSS 531.5
## 141 RightWhale_PA3_RUN4_GAM PA3 RUN4 GAM KAPPA 531.5
## 142 RightWhale_PA3_RUN4_RF PA3 RUN4 RF ROC 531.0
## 143 RightWhale_PA3_RUN4_RF PA3 RUN4 RF TSS 528.0
## 144 RightWhale_PA3_RUN4_RF PA3 RUN4 RF KAPPA 528.0
## 145 RightWhale_PA3_RUN5_GLM PA3 RUN5 GLM ROC 545.0
## 146 RightWhale_PA3_RUN5_GLM PA3 RUN5 GLM TSS 545.0
## 147 RightWhale_PA3_RUN5_GLM PA3 RUN5 GLM KAPPA 545.0
## 148 RightWhale_PA3_RUN5_GAM PA3 RUN5 GAM ROC 607.5
## 149 RightWhale_PA3_RUN5_GAM PA3 RUN5 GAM TSS 587.0
## 150 RightWhale_PA3_RUN5_GAM PA3 RUN5 GAM KAPPA 587.0
## 151 RightWhale_PA3_RUN5_RF PA3 RUN5 RF ROC 515.0
## 152 RightWhale_PA3_RUN5_RF PA3 RUN5 RF TSS 515.0
## 153 RightWhale_PA3_RUN5_RF PA3 RUN5 RF KAPPA 515.0
## 154 RightWhale_PA3_allRun_GLM PA3 allRun GLM ROC 527.5
## 155 RightWhale_PA3_allRun_GLM PA3 allRun GLM TSS 525.0
## 156 RightWhale_PA3_allRun_GLM PA3 allRun GLM KAPPA 525.0
## 157 RightWhale_PA3_allRun_GAM PA3 allRun GAM ROC 554.5
## 158 RightWhale_PA3_allRun_GAM PA3 allRun GAM TSS 551.5
## 159 RightWhale_PA3_allRun_GAM PA3 allRun GAM KAPPA 551.5
## 160 RightWhale_PA3_allRun_RF PA3 allRun RF ROC 526.0
## 161 RightWhale_PA3_allRun_RF PA3 allRun RF TSS 529.0
## 162 RightWhale_PA3_allRun_RF PA3 allRun RF KAPPA 529.0
## 163 RightWhale_PA4_RUN1_GLM PA4 RUN1 GLM ROC 343.5
## 164 RightWhale_PA4_RUN1_GLM PA4 RUN1 GLM TSS 341.0
## 165 RightWhale_PA4_RUN1_GLM PA4 RUN1 GLM KAPPA 636.0
## 166 RightWhale_PA4_RUN1_GAM PA4 RUN1 GAM ROC 584.0
## 167 RightWhale_PA4_RUN1_GAM PA4 RUN1 GAM TSS 582.0
## 168 RightWhale_PA4_RUN1_GAM PA4 RUN1 GAM KAPPA 582.0
## 169 RightWhale_PA4_RUN1_RF PA4 RUN1 RF ROC 505.0
## 170 RightWhale_PA4_RUN1_RF PA4 RUN1 RF TSS 505.0
## 171 RightWhale_PA4_RUN1_RF PA4 RUN1 RF KAPPA 505.0
## 172 RightWhale_PA4_RUN2_GLM PA4 RUN2 GLM ROC 375.5
## 173 RightWhale_PA4_RUN2_GLM PA4 RUN2 GLM TSS 371.0
## 174 RightWhale_PA4_RUN2_GLM PA4 RUN2 GLM KAPPA 757.0
## 175 RightWhale_PA4_RUN2_GAM PA4 RUN2 GAM ROC 553.0
## 176 RightWhale_PA4_RUN2_GAM PA4 RUN2 GAM TSS 553.0
## 177 RightWhale_PA4_RUN2_GAM PA4 RUN2 GAM KAPPA 553.0
## 178 RightWhale_PA4_RUN2_RF PA4 RUN2 RF ROC 502.0
## 179 RightWhale_PA4_RUN2_RF PA4 RUN2 RF TSS 505.0
## 180 RightWhale_PA4_RUN2_RF PA4 RUN2 RF KAPPA 505.0
## 181 RightWhale_PA4_RUN3_GLM PA4 RUN3 GLM ROC 395.0
## 182 RightWhale_PA4_RUN3_GLM PA4 RUN3 GLM TSS 395.0
## 183 RightWhale_PA4_RUN3_GLM PA4 RUN3 GLM KAPPA 395.0
## 184 RightWhale_PA4_RUN3_GAM PA4 RUN3 GAM ROC 591.0
## 185 RightWhale_PA4_RUN3_GAM PA4 RUN3 GAM TSS 589.0
## 186 RightWhale_PA4_RUN3_GAM PA4 RUN3 GAM KAPPA 589.0
## 187 RightWhale_PA4_RUN3_RF PA4 RUN3 RF ROC 535.0
## 188 RightWhale_PA4_RUN3_RF PA4 RUN3 RF TSS 530.0
## 189 RightWhale_PA4_RUN3_RF PA4 RUN3 RF KAPPA 530.0
## 190 RightWhale_PA4_RUN4_GLM PA4 RUN4 GLM ROC 394.5
## 191 RightWhale_PA4_RUN4_GLM PA4 RUN4 GLM TSS 395.0
## 192 RightWhale_PA4_RUN4_GLM PA4 RUN4 GLM KAPPA 704.0
## 193 RightWhale_PA4_RUN4_GAM PA4 RUN4 GAM ROC 578.0
## 194 RightWhale_PA4_RUN4_GAM PA4 RUN4 GAM TSS 567.0
## 195 RightWhale_PA4_RUN4_GAM PA4 RUN4 GAM KAPPA 567.0
## 196 RightWhale_PA4_RUN4_RF PA4 RUN4 RF ROC 533.0
## 197 RightWhale_PA4_RUN4_RF PA4 RUN4 RF TSS 530.0
## 198 RightWhale_PA4_RUN4_RF PA4 RUN4 RF KAPPA 530.0
## 199 RightWhale_PA4_RUN5_GLM PA4 RUN5 GLM ROC 419.5
## 200 RightWhale_PA4_RUN5_GLM PA4 RUN5 GLM TSS 421.0
## 201 RightWhale_PA4_RUN5_GLM PA4 RUN5 GLM KAPPA 697.0
## 202 RightWhale_PA4_RUN5_GAM PA4 RUN5 GAM ROC 510.0
## 203 RightWhale_PA4_RUN5_GAM PA4 RUN5 GAM TSS 508.0
## 204 RightWhale_PA4_RUN5_GAM PA4 RUN5 GAM KAPPA 550.0
## 205 RightWhale_PA4_RUN5_RF PA4 RUN5 RF ROC 517.0
## 206 RightWhale_PA4_RUN5_RF PA4 RUN5 RF TSS 515.0
## 207 RightWhale_PA4_RUN5_RF PA4 RUN5 RF KAPPA 515.0
## 208 RightWhale_PA4_allRun_GLM PA4 allRun GLM ROC 395.5
## 209 RightWhale_PA4_allRun_GLM PA4 allRun GLM TSS 395.0
## 210 RightWhale_PA4_allRun_GLM PA4 allRun GLM KAPPA 395.0
## 211 RightWhale_PA4_allRun_GAM PA4 allRun GAM ROC 566.5
## 212 RightWhale_PA4_allRun_GAM PA4 allRun GAM TSS 563.0
## 213 RightWhale_PA4_allRun_GAM PA4 allRun GAM KAPPA 563.0
## 214 RightWhale_PA4_allRun_RF PA4 allRun RF ROC 544.0
## 215 RightWhale_PA4_allRun_RF PA4 allRun RF TSS 540.0
## 216 RightWhale_PA4_allRun_RF PA4 allRun RF KAPPA 540.0
## 217 RightWhale_allData_allRun_GLM allData allRun GLM ROC 96.5
## 218 RightWhale_allData_allRun_GLM allData allRun GLM TSS 95.0
## 219 RightWhale_allData_allRun_GLM allData allRun GLM KAPPA 273.0
## 220 RightWhale_allData_allRun_GAM allData allRun GAM ROC 151.0
## 221 RightWhale_allData_allRun_GAM allData allRun GAM TSS 151.0
## 222 RightWhale_allData_allRun_GAM allData allRun GAM KAPPA 151.0
## 223 RightWhale_allData_allRun_RF allData allRun RF ROC 503.0
## 224 RightWhale_allData_allRun_RF allData allRun RF TSS 502.0
## 225 RightWhale_allData_allRun_RF allData allRun RF KAPPA 502.0
## sensitivity specificity calibration validation evaluation
## 1 98.361 83.571 0.946 0.970 0.971
## 2 98.361 83.571 0.819 0.850 0.962
## 3 95.082 86.429 0.759 0.774 0.870
## 4 96.721 70.714 0.815 0.830 0.505
## 5 96.721 70.714 0.674 0.717 0.462
## 6 96.721 70.714 0.571 0.605 0.516
## 7 100.000 100.000 1.000 0.994 0.981
## 8 100.000 100.000 1.000 0.950 0.962
## 9 100.000 100.000 1.000 0.920 0.870
## 10 95.082 88.571 0.969 0.935 0.971
## 11 95.082 88.571 0.837 0.663 0.962
## 12 86.885 95.000 0.823 0.687 0.870
## 13 98.361 72.143 0.825 0.807 0.505
## 14 98.361 72.143 0.705 0.612 0.462
## 15 95.082 75.000 0.610 0.512 0.516
## 16 100.000 100.000 1.000 0.983 0.971
## 17 100.000 100.000 1.000 0.890 0.962
## 18 100.000 100.000 1.000 0.890 0.870
## 19 100.000 85.714 0.958 0.943 0.971
## 20 100.000 85.714 0.857 0.773 0.962
## 21 93.443 90.000 0.798 0.864 0.870
## 22 100.000 73.571 0.825 0.746 0.582
## 23 100.000 73.571 0.736 0.606 0.462
## 24 100.000 73.571 0.628 0.512 0.516
## 25 100.000 100.000 1.000 0.981 0.971
## 26 100.000 100.000 1.000 0.851 0.962
## 27 100.000 100.000 1.000 0.861 0.870
## 28 98.361 85.000 0.958 0.960 0.971
## 29 98.361 85.000 0.834 0.778 0.962
## 30 86.885 92.857 0.790 0.760 0.870
## 31 98.361 73.571 0.827 0.798 0.505
## 32 98.361 72.857 0.712 0.645 0.462
## 33 98.361 72.857 0.608 0.539 0.516
## 34 100.000 100.000 1.000 0.978 0.981
## 35 100.000 100.000 1.000 0.912 0.962
## 36 100.000 100.000 1.000 0.892 0.870
## 37 91.803 92.143 0.964 0.946 0.971
## 38 98.361 85.000 0.834 0.762 0.962
## 39 90.164 92.857 0.815 0.724 0.870
## 40 98.361 72.857 0.833 0.782 0.505
## 41 98.361 72.143 0.705 0.628 0.462
## 42 98.361 72.143 0.600 0.520 0.516
## 43 100.000 100.000 1.000 0.976 0.971
## 44 100.000 100.000 1.000 0.890 0.923
## 45 100.000 100.000 1.000 0.890 0.762
## 46 98.851 83.500 0.958 NA 0.971
## 47 98.851 83.500 0.824 NA 0.962
## 48 87.356 93.000 0.796 NA 0.870
## 49 97.701 71.000 0.819 NA 0.505
## 50 97.701 71.000 0.687 NA 0.462
## 51 96.552 71.500 0.585 NA 0.516
## 52 100.000 100.000 1.000 NA 0.971
## 53 100.000 100.000 1.000 NA 0.962
## 54 100.000 100.000 1.000 NA 0.870
## 55 98.361 87.143 0.930 0.945 0.971
## 56 98.361 87.143 0.855 0.845 0.962
## 57 98.361 87.143 0.793 0.793 0.870
## 58 100.000 76.429 0.840 0.806 0.514
## 59 100.000 75.714 0.757 0.612 0.462
## 60 100.000 75.714 0.654 0.502 0.516
## 61 100.000 100.000 1.000 0.989 0.846
## 62 100.000 100.000 1.000 0.862 0.615
## 63 100.000 100.000 1.000 0.817 0.634
## 64 98.361 86.429 0.930 0.920 0.971
## 65 98.361 86.429 0.848 0.733 0.962
## 66 96.721 87.857 0.791 0.707 0.870
## 67 100.000 75.714 0.824 0.816 0.514
## 68 100.000 75.714 0.757 0.700 0.462
## 69 100.000 75.714 0.654 0.585 0.516
## 70 100.000 100.000 1.000 0.971 0.933
## 71 100.000 100.000 1.000 0.856 0.846
## 72 100.000 100.000 1.000 0.838 0.634
## 73 96.721 87.143 0.934 0.931 0.971
## 74 95.082 88.571 0.837 0.751 0.962
## 75 91.803 90.714 0.796 0.730 0.870
## 76 98.361 75.000 0.843 0.807 0.514
## 77 98.361 73.571 0.719 0.683 0.462
## 78 98.361 75.000 0.622 0.566 0.516
## 79 100.000 100.000 1.000 0.981 0.899
## 80 100.000 100.000 1.000 0.801 0.673
## 81 100.000 100.000 1.000 0.784 0.634
## 82 100.000 88.571 0.958 0.924 0.942
## 83 100.000 88.571 0.886 0.740 0.885
## 84 100.000 88.571 0.825 0.674 0.672
## 85 100.000 78.571 0.864 0.738 0.514
## 86 100.000 78.571 0.786 0.578 0.462
## 87 100.000 78.571 0.690 0.466 0.516
## 88 100.000 100.000 1.000 0.967 0.846
## 89 100.000 100.000 1.000 0.813 0.654
## 90 100.000 100.000 1.000 0.831 0.516
## 91 96.721 85.714 0.936 0.914 0.971
## 92 96.721 85.714 0.824 0.867 0.962
## 93 95.082 87.143 0.769 0.797 0.870
## 94 98.361 70.000 0.805 0.886 0.514
## 95 98.361 70.000 0.684 0.817 0.462
## 96 98.361 70.000 0.575 0.729 0.516
## 97 100.000 100.000 1.000 0.990 0.865
## 98 100.000 100.000 1.000 0.906 0.692
## 99 100.000 100.000 1.000 0.916 0.634
## 100 96.552 88.000 0.933 NA 0.971
## 101 96.552 87.500 0.841 NA 0.962
## 102 96.552 87.500 0.784 NA 0.870
## 103 98.851 74.000 0.828 NA 0.514
## 104 98.851 74.000 0.729 NA 0.462
## 105 98.851 74.000 0.625 NA 0.516
## 106 100.000 100.000 1.000 NA 0.841
## 107 100.000 100.000 1.000 NA 0.654
## 108 100.000 100.000 1.000 NA 0.516
## 109 98.361 89.286 0.950 0.905 0.971
## 110 98.361 89.286 0.876 0.795 0.962
## 111 98.361 89.286 0.823 0.724 0.870
## 112 100.000 78.571 0.848 0.751 0.519
## 113 98.361 80.000 0.784 0.683 0.462
## 114 98.361 80.000 0.697 0.566 0.516
## 115 100.000 100.000 1.000 0.950 0.966
## 116 100.000 100.000 1.000 0.785 0.923
## 117 100.000 100.000 1.000 0.760 0.762
## 118 96.721 87.857 0.945 0.923 0.971
## 119 96.721 87.857 0.846 0.850 0.962
## 120 96.721 87.857 0.791 0.774 0.870
## 121 100.000 76.429 0.829 0.796 0.514
## 122 100.000 76.429 0.764 0.717 0.462
## 123 98.361 77.857 0.669 0.597 0.516
## 124 100.000 100.000 1.000 0.928 0.962
## 125 100.000 100.000 1.000 0.768 0.923
## 126 100.000 100.000 1.000 0.736 0.762
## 127 98.361 85.714 0.954 0.942 0.971
## 128 91.803 92.143 0.839 0.571 0.962
## 129 91.803 92.143 0.817 0.597 0.870
## 130 100.000 71.429 0.787 0.903 0.505
## 131 100.000 71.429 0.714 0.812 0.462
## 132 100.000 71.429 0.603 0.746 0.516
## 133 100.000 100.000 1.000 0.957 0.971
## 134 100.000 100.000 1.000 0.703 0.923
## 135 100.000 100.000 1.000 0.718 0.762
## 136 96.721 86.429 0.936 0.936 0.971
## 137 96.721 86.429 0.831 0.845 0.962
## 138 96.721 86.429 0.771 0.793 0.870
## 139 98.361 75.000 0.826 0.826 0.514
## 140 98.361 75.000 0.734 0.800 0.462
## 141 98.361 75.000 0.634 0.707 0.516
## 142 100.000 100.000 1.000 0.956 0.962
## 143 100.000 100.000 1.000 0.703 0.923
## 144 100.000 100.000 1.000 0.718 0.762
## 145 96.721 86.429 0.935 0.954 0.971
## 146 96.721 86.429 0.831 0.900 0.962
## 147 96.721 86.429 0.771 0.845 0.870
## 148 100.000 78.571 0.829 0.787 0.514
## 149 100.000 77.857 0.779 0.683 0.462
## 150 100.000 77.857 0.681 0.566 0.516
## 151 100.000 100.000 1.000 0.989 0.962
## 152 100.000 100.000 1.000 0.933 0.923
## 153 100.000 100.000 1.000 0.894 0.762
## 154 96.552 88.000 0.943 NA 0.971
## 155 96.552 88.000 0.846 NA 0.962
## 156 96.552 88.000 0.792 NA 0.870
## 157 98.851 76.500 0.821 NA 0.514
## 158 98.851 76.500 0.754 NA 0.462
## 159 98.851 76.500 0.656 NA 0.516
## 160 100.000 100.000 1.000 NA 0.962
## 161 100.000 100.000 1.000 NA 0.923
## 162 100.000 100.000 1.000 NA 0.762
## 163 98.361 90.000 0.959 0.964 0.971
## 164 98.361 89.286 0.884 0.768 0.962
## 165 91.803 93.571 0.838 0.718 0.870
## 166 98.361 62.143 0.723 0.713 0.510
## 167 98.361 62.143 0.605 0.612 0.462
## 168 98.361 62.143 0.488 0.502 0.516
## 169 100.000 100.000 1.000 0.978 0.981
## 170 100.000 100.000 1.000 0.835 0.962
## 171 100.000 100.000 1.000 0.835 0.870
## 172 96.721 90.000 0.964 0.947 0.971
## 173 96.721 89.286 0.867 0.790 0.962
## 174 88.525 94.286 0.824 0.779 0.870
## 175 98.361 67.143 0.758 0.754 0.505
## 176 98.361 67.143 0.655 0.700 0.462
## 177 98.361 67.143 0.542 0.585 0.516
## 178 100.000 100.000 1.000 0.982 0.971
## 179 100.000 100.000 1.000 0.873 0.962
## 180 100.000 100.000 1.000 0.864 0.870
## 181 98.361 91.429 0.966 0.941 0.971
## 182 98.361 91.429 0.898 0.713 0.962
## 183 98.361 91.429 0.854 0.683 0.870
## 184 98.361 70.714 0.765 0.706 0.510
## 185 98.361 70.714 0.691 0.633 0.462
## 186 98.361 70.714 0.583 0.511 0.516
## 187 100.000 100.000 1.000 0.963 0.981
## 188 100.000 100.000 1.000 0.708 0.962
## 189 100.000 100.000 1.000 0.700 0.870
## 190 96.721 88.571 0.955 0.963 0.971
## 191 96.721 88.571 0.853 0.895 0.962
## 192 88.525 93.571 0.813 0.805 0.870
## 193 98.361 69.286 0.741 0.765 0.505
## 194 98.361 68.571 0.669 0.683 0.462
## 195 98.361 68.571 0.558 0.566 0.516
## 196 100.000 100.000 1.000 0.980 0.981
## 197 100.000 100.000 1.000 0.928 0.962
## 198 100.000 100.000 1.000 0.918 0.870
## 199 95.082 90.714 0.960 0.958 0.971
## 200 95.082 90.714 0.858 0.823 0.962
## 201 90.164 95.000 0.848 0.755 0.870
## 202 100.000 67.857 0.752 0.740 0.514
## 203 100.000 67.143 0.671 0.683 0.462
## 204 98.361 68.571 0.558 0.566 0.516
## 205 100.000 100.000 1.000 0.978 0.981
## 206 100.000 100.000 1.000 0.895 0.962
## 207 100.000 100.000 1.000 0.867 0.870
## 208 97.701 89.000 0.956 NA 0.971
## 209 97.701 89.000 0.867 NA 0.962
## 210 97.701 89.000 0.814 NA 0.870
## 211 98.851 69.000 0.749 NA 0.510
## 212 98.851 68.500 0.679 NA 0.462
## 213 98.851 68.500 0.566 NA 0.516
## 214 100.000 100.000 1.000 NA 0.981
## 215 100.000 100.000 1.000 NA 0.962
## 216 100.000 100.000 1.000 NA 0.870
## 217 97.701 85.786 0.956 NA 0.971
## 218 97.701 85.535 0.832 NA 0.962
## 219 88.506 92.830 0.656 NA 0.870
## 220 98.851 73.082 0.806 NA 0.505
## 221 98.851 73.082 0.719 NA 0.462
## 222 98.851 73.082 0.344 NA 0.516
## 223 100.000 100.000 1.000 NA 0.952
## 224 100.000 100.000 1.000 NA 0.923
## 225 100.000 100.000 1.000 NA 0.762# Select TSS
modelEvals <- modelEvals |>
dplyr::filter(metric.eval == "TSS") |>
dplyr::group_by(algo) |>
dplyr::reframe(TSS = evaluation,
name = full.name)
# Plot TSS
ggplot(data = modelEvals, mapping = aes(x = algo, y = TSS)) +
geom_boxplot() +
labs(x = "Model") +
theme_bw()
Variable contributions
The contributions of different variables are also in the output model object. Last time I checked the contributions were not relative, but that may have changed.
# Extract variable importance
varImportance <- get_variables_importance(obj = modelOut)
# Plot variable importance
ggplot(data = varImportance, mapping = aes(x = algo, y = var.imp)) +
geom_boxplot() +
facet_wrap(~expl.var) +
labs(x = "Model") +
theme_bw()
Response curves
Response curves show the predicted values of the response across the range of each environmental variable. Checking the response curves is a nice way to see what relationships the model may be picking up on, and to look at the ecological plausiblity of a model. If you get a really wacky response curve, it’s a good idea to go back and check that model. It might mean that the model does not meet the right statistical assumptions or a term is insignificant.
# Select highest performing GLMs
select_models <- modelEvals[order(-modelEvals$TSS),] |>
dplyr::filter(algo == "GLM") |>
head(5)
# GLM response curves
par(mar=c(3,3,3,3))
bm_PlotResponseCurves(bm.out = modelOut,
models.chosen = select_models$name,
new.env = get_formal_data(modelOut, "expl.var"),
show.variables = get_formal_data(modelOut, "expl.var.names"),
fixed.var = "mean",
do.bivariate = FALSE,
do.plot = TRUE,
do.progress = TRUE)## No id variables; using all as measure variables
# Select highest performing GAMs
select_models <- modelEvals[order(-modelEvals$TSS),] |>
dplyr::filter(algo == "GAM") |>
head(5)
# GAM response curves
par(mar=c(3,3,3,3))
bm_PlotResponseCurves(bm.out = modelOut,
models.chosen = select_models$name,
new.env = get_formal_data(modelOut, "expl.var"),
show.variables = get_formal_data(modelOut, "expl.var.names"),
fixed.var = "mean",
do.bivariate = FALSE,
do.plot = TRUE,
do.progress = TRUE)## No id variables; using all as measure variables
# Select highest performing RFs
select_models <- modelEvals[order(-modelEvals$TSS),] |>
dplyr::filter(algo == "GLM") |>
head(5)
# RF response curves
par(mar=c(3,3,3,3))
bm_PlotResponseCurves(bm.out = modelOut,
models.chosen = select_models$name,
new.env = get_formal_data(modelOut, "expl.var"),
show.variables = get_formal_data(modelOut, "expl.var.names"),
fixed.var = "mean",
do.bivariate = FALSE,
do.plot = TRUE,
do.progress = TRUE)## No id variables; using all as measure variables
Project models onto summer 2015
Finally, we will project our model onto the environmental data from 2015 (the year we left out). To “quality-control” the projections we will only use the top 5 performing models for each algorithm. This is also a good chance to see how predictions from different model formulas differ.
# Project models onto year 2015 (witheld from model)
env_proj <- raster::subset(env_dat, c(grep('2015', names(env_dat))))
names(env_proj) <- vars
# Select highest performing GLMs
select_models <- modelEvals[order(-modelEvals$TSS),] |>
dplyr::filter(algo == "GLM") |>
head(5)
glmProj <- BIOMOD_Projection(bm.mod = modelOut,
new.env = env_proj,
proj.name = "GLM",
models.chosen = select_models$name,
metric.binary = "TSS",
compress = TRUE)##
## -=-=-=-=-=-=-=-=-=-=-=-=-= Do Single Models Projection -=-=-=-=-=-=-=-=-=-=-=-=-=
##
## > Building clamping mask
##
## > Projecting RightWhale_PA1_RUN1_GLM ...
## > Projecting RightWhale_PA1_RUN2_GLM ...
## > Projecting RightWhale_PA1_RUN3_GLM ...
## > Projecting RightWhale_PA1_RUN4_GLM ...
## > Projecting RightWhale_PA1_RUN5_GLM ...
##
## > Building TSS binaries / filtered
##
## -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Done -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=# Select highest performing GAMs
select_models <- modelEvals[order(-modelEvals$TSS),] |>
dplyr::filter(algo == "GAM") |>
head(5)
gamProj <- BIOMOD_Projection(bm.mod = modelOut,
new.env = env_proj,
proj.name = "gam",
models.chosen = select_models$name,
metric.binary = "TSS",
compress = TRUE)##
## -=-=-=-=-=-=-=-=-=-=-=-=-= Do Single Models Projection -=-=-=-=-=-=-=-=-=-=-=-=-=
##
## > Building clamping mask
##
## > Projecting RightWhale_PA1_RUN1_GAM ...
## > Projecting RightWhale_PA1_RUN2_GAM ...
## > Projecting RightWhale_PA1_RUN3_GAM ...
## > Projecting RightWhale_PA1_RUN4_GAM ...
## > Projecting RightWhale_PA1_RUN5_GAM ...
##
## > Building TSS binaries / filtered
##
## -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Done -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=# Select highest performing RFs
select_models <- modelEvals[order(-modelEvals$TSS),] |>
dplyr::filter(algo == "RF") |>
head(5)
rfProj <- BIOMOD_Projection(bm.mod = modelOut,
new.env = env_proj,
proj.name = "RF",
models.chosen = select_models$name,
metric.binary = "TSS",
compress = TRUE)##
## -=-=-=-=-=-=-=-=-=-=-=-=-= Do Single Models Projection -=-=-=-=-=-=-=-=-=-=-=-=-=
##
## > Building clamping mask
##
## > Projecting RightWhale_PA1_RUN1_RF ...
## > Projecting RightWhale_PA1_RUN2_RF ...
## > Projecting RightWhale_PA1_RUN3_RF ...
## > Projecting RightWhale_PA1_RUN4_RF ...
## > Projecting RightWhale_PA1_allRun_RF ...
##
## > Building TSS binaries / filtered
##
## -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Done -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=plot(glmProj)## Loading required namespace: tidyterra
plot(gamProj)
plot(rfProj)
Looking a little further into the projection objects (using GAM as an example):
# Look closer at GAM projection
gamProj##
## -=-=-=-=-=-=-=-=-=-=-=-=-=-= BIOMOD.projection.out -=-=-=-=-=-=-=-=-=-=-=-=-=-=
##
## Projection directory : ./RightWhale/gam
##
##
## sp.name : RightWhale
##
## expl.var.names : Depth SST CHL
##
##
## modeling.id : RightWhale ( ./RightWhale/RightWhale.RightWhale.models.out )
##
## models.projected :
## RightWhale_PA1_RUN1_GAM, RightWhale_PA1_RUN2_GAM, RightWhale_PA1_RUN3_GAM, RightWhale_PA1_RUN4_GAM, RightWhale_PA1_RUN5_GAM
##
## available binary projection : TSS
##
## -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=fp <- gamProj@proj.out@link[1]
# Load in as SpatRaster
gamStack <- terra::rast(fp)
# fetch right whale sightings
whales2015 <- RightWhale |>
dplyr::filter(year == 2015)
# Plot highest performing GAM (slot 1)
plot(gamStack$RightWhale_PA1_RUN1_GAM)
points(x = whales2015$lon, y = whales2015$lat, cex = 2, pch = 16) # Add whale sightings